Combining STPA and BDD for Safety Analysis and Verification in Agile Development: A Controlled Experiment

Context: Agile development is in widespread use, even in safety-critical domains. Motivation: However, there is a lack of an appropriate safety analysis and verification method in agile development. Objective: In this paper, we investigate the use of Behavior Driven Development (BDD) instead of standard User Acceptance Testing (UAT) for safety verification with System-Theoretic Process Analysis (STPA) for safety analysis in agile development. Method: We evaluate the effect of this combination in a controlled experiment with 44 students in terms of productivity, test thoroughness, fault detection effectiveness and communication effectiveness. Results: The results show that BDD is more effective for safety verification regarding the impact on communication effectiveness than standard UAT, whereas productivity, test thoroughness and fault detection effectiveness show no statistically significant difference in our controlled experiment. Conclusion: The combination of BDD and STPA seems promising with an enhancement on communication, but its impact needs more research.